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DETAILED ACTION 



Response to Amendment 



1. 



Claims 53- 



■77 are pending. 



2. 



Claims 53- 



■59 have been withdrawn. 



3. 



Claims 60- 



■77 are considered for examination. 



4. 



Claims 60, 



67, 73-77 have been amended. 



Continued Examination Under 37 CFR 1. 1 14 



5. A request for continued examination under 37 CFR 1.114, Including the fee set 
forth In 37 CFR 1 .17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.1 14. Applicant's submission filed on 
6/6/201 1 has been entered. 



6. Applicant argues "Amended claim 60 recites, in part, "selecting a category of 
additional information related to the type of sentence" and "adding the additional 
information to the Inputted text with a change processing device so that the additional 
information is added before the beginning of the inputted text or after the end of the 
inputted text." (Emphasis added). Although of different scope, amended claims 67, 73, 
and 74 recite similar features. Applicants submit that Papineni does not disclose each 



Response to Arguments 
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and every element of at least these claims." (Remarks, Page 8, H 3) The Examiner 
disagrees. After further consideration of the Papineni reference, Papineni teaches the 
claim language as amended. Papineni, column 15, lines 54-67, selects additional 
information by providing specific confirmations of the index or fund, ...confirming transfer 
of five thousand dollars from fidelity magellan to vanguard index trust fund five hundred, 
please say yes or no... The selected additional information is the type of action and 
fund for the category of additional information (confirmation). Additionally, note the 
confirmation is in two forms, first being the "confirming transfer" at the beginning of the 
response, second being "please say yes or no" at the end of the response. Therefore, 
Papineni teaches the claim language as amended. 

7. Applicant further argues "Papineni is directed to a "system for conversant 
interaction" and a "dialog manager having task-oriented forms." See Papineni, Abstract, 
col. 3, lines 8-9. Papineni further provides an interaction between the "conversant 
interaction" system and a human user, see Papineni, col. 15, but as indicated in the 
Advisory Action, the reference generates a "semantic representation" of the "broken 
down" input that is used to generate a "suitable response." See Advisory Action, page 3. 
In response to an input of "1 would like to transfer five thousand dollars," the response 
is "confirming transfer of five thousand dollars from fidelity magellan to vanguard index 
trust fund five hundred. Please say yes or no." See Papineni, col. 15, lines 59-61 . As 
such, the system of Papineni has "broken down" the inputted text but does not disclose 
selecting "additional information related to the type of sentence" or adding "additional 
information. ..before the beginning of the inputted text or after the end of the inputted 
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text," as claimed. (Emphasis added)." (Remarks, Pages 8-9) The Examiner disagrees 
for the same reasons as set forth above in item 4. 

8. Applicant further argues "Amended claim 60 recites, in part, "selecting a category 
of additional information related to the type of sentence" and "adding the additional 
information to the inputted text with a change processing device so that the additional 
information is added before the beginning of the inputted text or after the end of the 
inputted text." (Emphasis added). Although of different scope, amended claims 67, 73, 
and 74 recite similar features. Applicants submit that Gabai and Papineni, either alone 
or in combination, do not disclose or suggest each and every element of at least these 
claims." (Remarks, Page 9, H 4) The Examiner disagrees for the same reasons as set 
forth above in item 4. Papineni teaches the amended claim language, therefore the 
combination of Gabai in view of Papineni also teaches the amended claim language. 

9. Applicant further argues "Gabai is directed to an interactive toy that can read 
inputted text, translate it, and explain the cultural significance of the text. See Gabai, 
col. 43, lines 20-34. Although Gabai provides for "translating a menu" then explaining 
the "cultural significance of the dishes," see Gabai, col. 43, lines 46-47, outputting a text 
translation is not the same as outputting inputted text. Further, the reference is silent 
with regard to "selecting a category of additional information related to the type of 
sentence" and "additional information is added before the beginning of the inputted text 
or after the end of the inputted text." (Emphasis added). Applicant respectfully submits 
that Papineni fails to make up for such deficiencies. Accordingly, Applicants respectfully 
request that this rejection be withdrawn." (Remarks, Pages 9-10) The Examiner notes 
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that the previous rejections were made through a combination of Gabai in view of 
Papineni, not singularly upon Gabai. As is shown above, Papineni provides the claim 
language as amended and therefore the Examiner disagrees for the same reasons as 
set forth above in item 4. 

1 0. The arguments on page 1 0, If 2- f 3 stem from the same arguments as 
presented above. Refer to the above item 4 as to why the combinations of Gabai in view 
of Papineni in view of Uwakubo and Papineni in view of McAllister teach the amended 
claim language. 

Information Disclosure Statement 

1 1 . The Information Disclosure Statement (IDS) submitted on 5/31/201 1 is in 
compliance with the provisions of 37 CFR 1 .97. 

Specification 

1 2. Applicant is reminded of the proper language and format for an abstract of the 
disclosure. 

The abstract should be in narrative form and generally limited to a single 
paragraph on a separate sheet within the range of 50 to 150 words. It is important that 
the abstract not exceed 1 50 words in length since the space provided for the abstract 
on the computer tape used by the printer is limited. The form and legal phraseology 
often used in patent claims, such as "means" and "said," should be avoided. The 
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abstract should describe the disclosure sufficiently to assist readers in deciding whether 
there is a need for consulting the full patent text for details. 

The language should be clear and concise and should not repeat information 
given in the title. It should avoid using phrases which can be implied, such as, "The 
disclosure concerns," "The disclosure defined by this invention," "The disclosure 
describes," etc. 



Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351 (a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21 (2) 
of such treaty in the English language. 

1 3. Claims 60, and 73-74 are rejected under 35 U.S.C. 1 02(e) as being anticipated 
by Papineni et al. (US Patent #6246981). 



As per claim 60, Papineni teaches the method comprising receiving inputted text 
with an information processing device, the inputted text including a sentence (Papineni, 
column 1 , lines 22-30, . . . The input and output could be either text-oriented or speech- 
oriented. Speech-oriented systems have a speech recognition subsystem (speech-to- 
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text system) and a speech synthesis subsystem (text-to-speech system). . . column 1 5, 
lines 5-67 further teach input sentences.); 

analyzing the inputted sentence with an information analysis device to determine 
information to be added comprising the steps of (Papineni, column 3, lines 8-19, ...A 
system for conversant interaction includes a recognizer for receiving and processing 
input information and outputting a recognized representation of the input information. A 
dialog manager is coupled to the recognizer for receiving the recognized representation 
of the input information, the dialog manager having task-oriented forms for associating 
user input information therewith, the dialog manager being capable of selecting an 
applicable form from the task-oriented forms responsive to the input information...): 

classifying the inputted sentence as one of a plurality of types of sentences, the 
plurality of types of sentences including a question and an explanation (Papineni, 
column 1 5, lines 1 0-31 , the example shows input questions and explanations, see 
. . . how about the vanguard index. . . (question), and . . ./ want to buy one hundred 
shares... (explanation). Additionally, column 15, lines 54-67 notes that the user asks "I 
would like to transfer five thousand dollars" (explanation), which includes the sentence 
"transfer five thousand dollars" which is further included in the response (as a 
confirmation question) below on lines 59-62.); 

selecting a category of additional information related to the type of sentence, the 
category being an expression which is suitable to the type of sentence (Papineni, 
column 15, lines 19-31, shows selection of additional information related to the type of 
sentence through a confirmation, ...confirming purchase of one hundred Shares of 
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vanguard index trust total stock market institutional shares, please say yes or no (fund 
name from context)... Column 15, lines 54-67 also show this confirmation of a transfer.); 
and 

selecting additional information in the selected category (Papineni, column 15, 
lines 54-67, selects additional information by providing specific confirmations of the 
index or fund, ...confirming transfer of five thousand dollars from fidelity magellan to 
vanguard index trust fund five hundred, please say yes or no. . . The selected additional 
information is the type of action and fund for the category of additional information 
(confirmation). Additionally, note the confirmation is in two forms, first being the 
"confirming transfer" at the beginning of the response, second being "please say yes or 
no" at the end of the response.); and 

adding the additional information to the inputted text with a change processing 
device so that the additional information is added before the beginning of the inputted 
text or after the end of the inputted text (Papineni, column 15, lines 54-67, ...confirming 
transfer of five thousand dollars from fidelity magellan to vanguard index trust fund five 
hundred, please say yes or no... The confirmation (additional information added to the 
input text of "transfer five thousand dollars") is in two forms, first being the "confirming 
transfer" at the beginning of the response, second being "please say yes or no" at the 
end of the response.); and 

outputting the inputted text to which the information is added with an information 
reproducing device (Papineni, column 15, lines 54-67 teaches the dialog interaction 
including the confirmation that is added to the input text. Further see column 1 , lines 22- 
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30, . . . The input and output could be either text-oriented or speech-oriented. Speech- 
oriented systems have a speech recognition subsystem (speech-to-text system) and a 
speech synthesis subsystem (text-to-speech system)... Column 15, lines 5-67 further 
teach input sentences. The information reproducing device is interpreted as the text to 
speech synthesizer.). 

As per claim 73, Papineni teaches the computer readable medium storing 
computer executable instructions for performing a method on a computer (Papineni, 
column 7, lines 41 -54 teaches a computer having a processor and memory) comprising: 

analyzing a sentence in inputted text to determine information to be added 
comprising the steps of (Papineni, column 3, lines 8-19, ...A system for conversant 
interaction includes a recognizer for receiving and processing input information and 
outputting a recognized representation of the input information. A dialog manager is 
coupled to the recognizer for receiving the recognized representation of the input 
information, the dialog manager having task-oriented forms for associating user input 
information therewith, the dialog manager being capable of selecting an applicable form 
from the task-oriented forms responsive to the input information. . .): 

classifying the inputted sentence as one of a plurality of types of sentences, the 
plurality of types of sentences including a question and an explanation (Papineni, 
column 15, lines 10-31, the example shows input questions and explanations, see 
. . . how about the vanguard index. . . (question), and . . . / want to buy one hundred 
shares... (explanation). Additionally, column 15, lines 54-67 notes that the user asks "I 
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would like to transfer five thousand dollars" (explanation), which includes the sentence 
"transfer five thousand dollars" which is further included in the response (as a 
confirmation question) below on lines 59-62.); 

selecting a category of additional information related to the type of sentence, the 
category being an expression which is suitable to the type of sentence (Papineni, 
column 15, lines 19-31, shows selection of additional information related to the type of 
sentence through a confirmation, ...confirming purchase of one hundred Shares of 
vanguard index trust total stock market institutional shares, please say yes or no (fund 
name from context)... Column 15, lines 54-67 also show this confirmation of a transfer.); 
and 

selecting additional information in the selected category (Papineni, column 15, 
lines 54-67, selects additional information by providing specific confirmations of the 
index or fund, ...confirming transfer of five thousand dollars from fidelity magellan to 
vanguard index trust fund five hundred, please say yes or no... The selected additional 
information is the type of action and fund for the category of additional information 
(confirmation). Additionally, note the confirmation is in two forms, first being the 
"confirming transfer" at the beginning of the response, second being "please say yes or 
no" at the end of the response.); and 

adding the additional information before the beginning of the inputted text or after 
the end of the inputted text (Papineni, column 15, lines 54-67, ...confirming transfer of 
five thousand dollars from fidelity magellan to vanguard index trust fund five hundred, 
please say yes or no... The confirmation (additional information added to the input text 
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of "transfer five tliousand dollars") is in two forms, first being the "confirming transfer" at 
the beginning of the response, second being "please say yes or no" at the end of the 
response.); and 

converting inputted text which the additional information is added, to voice 
(Papineni, column 15, lines 54-67 teaches the dialog interaction including the 
confirmation that is added to the input text. Further see column 1 , lines 22-30, ...The 
input and output could be either text-oriented or speech-oriented. Speech-oriented 
systems have a speech recognition subsystem (speech-to-text system) and a speech 
synthesis subsystem (text-to-speech system)... Column 15, lines 5-67 further teach 
input sentences. The information reproducing device is interpreted as the text to speech 
synthesizer.). 

As per claim 74, Papineni teaches the terminal (Papineni, column 7, lines 41-54 
teaches a computer having a processor and memory) comprising: 

an information processing device for receiving inputted text, having an 
information changing unit for analyzing a sentence in the inputted text to determine 
information to be added comprising the steps of (Papineni, column 3, lines 8-19, ...A 
system for conversant interaction includes a recognizer for receiving and processing 
input information and outputting a recognized representation of the input information. A 
dialog manager is coupled to the recognizer for receiving the recognized representation 
of the input information, the dialog manager having task-oriented forms for associating 
user input information therewith, the dialog manager being capable of selecting an 
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applicable form from the task-oriented forms responsive to ttie input information. . . The 
information processing device for receiving inputted text is interpreted as the dialog 
manager that accepts the text from the recognizer. The information changing unit is also 
supplied through the dialog manager to determine the appropriate response (analyzing 
the sentence input to determine information to be added).): 

classifying the sentence as one of a plurality of types of sentences, the plurality 
of types of sentences including a question and an explanation ((Papineni, column 15, 
lines 10-31, the example shows input questions and explanations, see ...how about the 
vanguard index... (question), and ...i want to buy one hundred shares... (explanation). 
Additionally, column 15, lines 54-67 notes that the user asks "I would like to transfer five 
thousand dollars" (explanation), which includes the sentence "transfer five thousand 
dollars" which is further included in the response (as a confirmation question) below on 
lines 59-62); 

selecting a category of additional information related to the type of sentence, the 
category being an expression which is suitable to the type of sentence (Papineni, 
column 15, lines 19-31, shows selection of additional information related to the type of 
sentence through a confirmation, ...confirming purchase of one hundred Shares of 
vanguard index trust total stock market institutional shares, please say yes or no (fund 
name from context)... Column 15, lines 54-67 also show this confirmation of a transfer.); 
and 

selecting additional information in the selected category and adding the 
information before the beginning of the inputted text or after the end of the inputted text 
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(Papineni, column 15, lines 54-67, selects additional information by providing specific 
confirmations of the index or fund, ...confirming transfer of five thousand dollars from 
fidelity magellan to vanguard index trust fund five hundred, please say yes or no. . . The 
selected additional information is the type of action and fund for the category of 
additional Information (confirmation). Additionally, note the confirmation Is In two forms, 
first being the "confirming transfer" at the beginning of the response, second being 
"please say yes or no" at the end of the response.); and 

an information reproducing device for converting an output from the information 
changing unit to voice (Papineni, column 15, lines 54-67 teaches the dialog Interaction 
Including the confirmation that is added to the input text. Further see column 1 , lines 22- 
30, . . . The input and output could be either text-oriented or speech-oriented. Speech- 
oriented systems have a speech recognition subsystem (speech-to-text system) and a 
speech synthesis subsystem (text-to-speech system)... Column 15, lines 5-67 further 
teach Input sentences. The information reproducing device is interpreted as the text to 
speech synthesizer.). 

Claim Rejections - 35 USC § 103 

The following Is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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1 4. Claims 60-64, 66-70, 72-75 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable by Gabai et al. (US Patent #6773344 hereinafter Gabai) in view of 
Papineni et al. (US Patent #6246981) 

As per claim 60, Gabai teaches the method comprising receiving inputted text 
with an information processing device (Gabai, column 43, lines 20-34, the toy can read 
text for translation or speech synthesis.); 

analyzing the inputted text with an information analysis device, the inputted text 
including a sentence to determine information to be added comprising the steps of 
(Gabai, column 43, lines 3-1 9, ...It is preferred, in such cases, that a toy not merely 
translate but combine its translations with other types of content that is appropriate to 
the given situation... Gabai teaches the ability to read local or ancient languages, 
column 43, lines 20-34 where the scanner is an information analysis unit. Information to 
be added is appropriate to a given situation. Column 43, lines 35-43 teaches that the 
input may be full sentences): 

adding the additional information to the inputted text with a change processing 
device (Gabai, column 43, lines 3-19, ...It is preferred, in such cases, that a toy not 
merely translate but combine its translations with other types of content that is 
appropriate to the given situation... Furthermore, column 43, lines 35-50 gives 
examples, one of which is that the user inputs a menu and then toy subsequently 
explains the cultural significance of the dishes. Figs .14 and 70 show that the output can 
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be processed an generated from the internal toy process or an external computer/server 
depending on the complexity of the input/operation.); 

outputting the inputted text to which the information is added with an information 
reproducing device (Gabai, column 53, lines 26-36, ...Their response includes, but is 
not limited to sound (including voice)... Fig. 66, 8085). 

Gabai fails to specifically teach classifying the inputted sentence as one of a 
plurality of types of sentences, the plurality of types of sentences including a question 
and an explanation; selecting a category of additional information related to the type of 
sentence, the category being an expression which is suitable to the type of sentence; 
selecting additional information in the selected category; and adding the additional 
information so that the additional information is added before the beginning of the 
inputted text or after the end of the inputted text. 

However, Papineni teaches classifying the inputted sentence as one of a plurality 
of types of sentences, the plurality of types of sentences including a question and an 
explanation (Papineni, column 15, lines 10-31 , the example shows input questions and 
explanations, see ... how about the vanguard index... (question), and ...i want to buy 
one hundred shares... (explanation). Additionally, column 15, lines 54-67 notes that the 
user asks "I would like to transfer five thousand dollars" (explanation), which includes 
the sentence "transfer five thousand dollars" which is further included in the response 
(as a confirmation question) below on lines 59-62.); 
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selecting a category of additional information related to the type of sentence, the 
category being an expression which is suitable to the type of sentence (Papineni, 
column 15, lines 19-31, shows selection of additional information related to the type of 
sentence through a confirmation, ...confirming purchase of one hundred Shares of 
vanguard index trust total stock market institutional shares, please say yes or no (fund 
name from context)... Column 15, lines 54-67 also show this confirmation of a transfer.); 
and 

selecting additional information in the selected category (Papineni, column 15, 
lines 54-67, selects additional information by providing specific confirmations of the 
index or fund, ...confirming transfer of five thousand dollars from fidelity magellan to 
vanguard index trust fund five hundred, please say yes or no. . . The selected additional 
information is the type of action and fund for the category of additional information 
(confirmation). Additionally, note the confirmation is in two forms, first being the 
"confirming transfer" at the beginning of the response, second being "please say yes or 
no" at the end of the response.); and 

adding the additional information to the inputted text with a change processing 
device so that the additional information is added before the beginning of the inputted 
text or after the end of the inputted text (Papineni, column 15, lines 54-67, ...confirming 
transfer of five thousand dollars from fidelity magellan to vanguard index trust fund five 
hundred, please say yes or no... The confirmation (additional information added to the 
input text of "transfer five thousand dollars") is in two forms, first being the "confirming 
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transfer" at the beginning of the response, second being "please say yes or no" at the 
end of the response.). 

Gabai and Papineni are analogous art because both deal with dialog interaction 
between a human and machine. Therefore, It would have been obvious to someone of 
ordinary skill in the art at the time of the invention to apply the 
confirmations/clarifications of Papineni with the base process of dialog interaction in 
Gabai because the result would have been predictable in providing a machine/user 
dialog interaction where the machine and user have a mutual understanding of the 
current dialog state. Therefore, it would have been obvious to someone of ordinary skill 
in the art at the time of the invention was made to combine Papineni with Gabai. 

As per claim 61 , claim 60 is incorporated and the combination of Gabai and 
Papineni teaches wherein the inputted text is translation text that is translated from a 
first language to a second language with an automatic interpretation device (Gabai, 
column 43, lines 3-19, Figs. 58 A-B teach that the toy interprets the scanned information 
in a language not native to the user for the user's understanding.). 

As per claim 62, claim 60 is incorporated and the combination of Gabai and 
Papineni teaches wherein a voice synthesis device converts the inputted text to which 
the information is added to a voice signal and outputs the voice signal (Gabai, column 
20, lines 14-36, ...transfer information to the user through sound (possibly using text-to- 
speech technology). ..). 
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As per claim 63, claim 60 is incorporated and the combination of Gabai and 
Papineni teaches wherein amount of information to be added is determined on the basis 
of an analysis result (Gabai, column 43, lines 44-50, ...translating an ancient inscription 
a toy offers its user a historical commentary on the period and the occasion on which it 
was written and the subjects it concerns..., There is inherently a determined amount of 
available additional information because the database stores available additional 
information in the database that is retrieved based upon the analysis.). 

As per claim 64, claim 60 is incorporated and the combination of Gabai and 
Papineni teaches where the information is prestored corresponding to a keyword 
(Gabai, column 46, lines 40-67, the toy listens for keywords in its analysis to understand 
the input and produce the appropriate response. Also, example II (columns 45-46) 
shows that the Information is related to the input keywords.). 

As per claim 66, claim 62 is incorporated and the combination of Gabai and 
Papineni teaches wherein the information is information for prompting a target (Gabai, 
columns 45-46, Example II, teaches that information is added for prompting a target 
using voice, column 46, lines 1-5). 

As per claim 67, Gabai teaches an information processing system (Gabai, 
column 16, lines 7-30) comprising: 
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an information processing device for receiving inputted text including a sentence, 
having an information clianging unit for analyzing the sentence to determine information 
to be added comprising the steps of (Gabai, column 43, lines 20-34, A server 
(information processing device for receiving inputted text) may receive the scanned 
information to translate (information changing unit supplied by server) the message with 
greater computing power. Further, Gabai, column 43, lines 3-19, ...It is preferred, in 
such cases, that a toy not merely translate but combine its translations with other types 
of content that is appropriate to the given situation... Therefore, the translating unit may 
also add translation content information. Furthermore, column 43, lines 35-50 gives 
examples, one of which is that the user inputs a menu and then toy subsequently 
explains the cultural significance of the dishes.): 

adding the additional information to text (Gabai, column 43, lines 3-19, ...It is 
preferred, in such cases, that a toy not merely translate but combine its translations with 
other types of content that is appropriate to the given situation... Furthermore, column 
43, lines 35-50 gives examples, one of which is that the user inputs a menu and then 
toy subsequently explains the cultural significance of the dishes. Figs .14 and 70 show 
that the output can be processed an generated from the internal toy process or an 
external computer/server depending on the complexity of the input/operation.); 

and an information reproducing device for converting and output from the 
information changing unit to voice (Gabai, column 53, lines 26-36, ...Their response 
includes, but is not limited to sound (including voice)...). 
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Gabai fails to teacli classifying the sentence as one of a plurality of types of 
sentences, the plurality of types of sentences including a question and an explanation; 
selecting a category of additional information related to the type of sentence, the 
category being an expression which is suitable to the type of sentence; and selecting 
additional information in the selected category and adding the additional information 
before the beginning of the inputted text or after the end of the inputted text; and an 
information reproducing device for converting an output from the information changing 
unit to voice. 

However, Papineni teaches classifying the sentence as one of a plurality of types 
of sentences, the plurality of types of sentences including a question and an explanation 
(Papineni, column 1 5, lines 1 0-31 , the example shows input questions and 
explanations, see ...how about the vanguard index... (question), and ...i want to buy 
one hundred shares... (explanation). Additionally, column 15, lines 54-67 notes that the 
user asks "I would like to transfer five thousand dollars" (explanation), which includes 
the sentence "transfer five thousand dollars" which is further included in the response 
(as a confirmation question) below on lines 59-62.); 

selecting a category of additional information related to the type of sentence, the 
category being an expression which is suitable to the type of sentence (Papineni, 
column 15, lines 19-31, shows selection of additional information related to the type of 
sentence through a confirmation, ...confirming purchase of one hundred Shares of 
vanguard index trust total stock market institutional shares, please say yes or no (fund 
name from context)... Column 15, lines 54-67 also show this confirmation of a transfer.); 
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selecting additional information in the selected category and adding the 
additional information before the beginning of the inputted text or after the end of the 
inputted text (Papineni, column 15, lines 54-67, selects additional information by 
providing specific confirmations of the index or fund, ...confirming transfer of five 
thousand dollars from fidelity magellan to vanguard index trust fund five hundred, 
please say yes or no... The selected additional information is the type of action and 
fund for the category of additional information (confirmation). Additionally, note the 
confirmation is in two forms, first being the "confirming transfer" at the beginning of the 
response, second being "please say yes or no" at the end of the response.); and 

Gabai and Papineni are analogous art because both deal with dialog interaction 
between a human and machine. Therefore, It would have been obvious to someone of 
ordinary skill in the art at the time of the invention to apply the 
confirmations/clarifications of Papineni with the base process of dialog interaction in 
Gabai because the result would have been predictable in providing a machine/user 
dialog interaction where the machine and user have a mutual understanding of the 
current dialog state. Therefore, it would have been obvious to someone of ordinary skill 
in the art at the time of the invention was made to combine Papineni with Gabai. 

As per claim 68, claim 67 is incorporated and the combination of Gabai and 
Papineni teaches further comprising an interpretation unit for translating the inputted 
text from a first language to a second language and outputting the translated text to the 
information changing unit (Gabai, column 43, lines 3-19, Figs. 58 A-B teach that the toy 
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interprets the scanned information tlirough a translation server (interpretation unit) tiiat 
can add (information clianging unit) content information, in a language not native to the 
user for the user's understanding.). 

As per claim 69, claim 67 is incorporated and the combination of Gabai and 
Papineni teaches wherein amount of information to be added is determined on the basis 
of an analysis result (Gabai, column 43, lines 44-50, ...translating an ancient inscription 
a toy offers its user a historical commentary on the period and the occasion on which it 
was written and the subjects it concerns. .., There is inherently a determined amount of 
available additional information because the database stores available additional 
information in the database that is retrieved based upon the analysis.). 

As per claim 70, claim 67 is incorporated and the combination of Gabai and 
Papineni teaches wherein the information changing unit comprises a memory device for 
storing the information corresponding to a keyword, extracts the keyword from the 
inputted text and selects the information stored into the memory device on the basis of 
the extracted keyword (Gabai, column 46, lines 40-67, the toy listens for keywords in its 
analysis to understand the input and produce the appropriate response. Also, example 
II (columns 45-46) shows that the information is related to the input keywords. Gabai, 
column 16, lines 7-30 further discloses the toy as a computational device which requires 
a memory for storing the input information for processing.). 
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As per claim 72, claim 66 is incorporated and the combination of Gabai and 
Papineni teaches wherein the information is information for prompting a target (Gabai, 
columns 45-46, Example II, teaches that information is added for prompting a target 
using voice, column 46, lines 1-5). 

As per claim 73, Gabai teaches a non-transitory computer readable medium 
storing computer instructions for execution of a method comprising (Gabai, column 16, 
lines 7-30 and column 69, lines 12-15): 

analyzing the inputted text with an information analysis device, the inputted text 
including a sentence to determine information to be added comprising the steps of 
(Gabai, column 43, lines 3-1 9, ...It is preferred, in such cases, that a toy not merely 
translate but combine its translations with other types of content that is appropriate to 
the given situation... Gabai teaches the ability to read local or ancient languages, 
column 43, lines 20-34 where the scanner is an information analysis unit. Information to 
be added is appropriate to a given situation. Column 43, lines 35-43 teaches that the 
input may be full sentences): 

adding the additional information to text (Gabai, column 43, lines 3-19, ...It is 
preferred, in such cases, that a toy not merely translate but combine its translations with 
other types of content that is appropriate to the given situation. . . Furthermore, column 
43, lines 35-50 gives examples, one of which is that the user inputs a menu and then 
toy subsequently explains the cultural significance of the dishes. Figs .14 and 70 show 
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that the output can be processed an generated from the internal toy process or an 
external computer/server depending on the complexity of the input/operation.); 

converting inputted text which the information is added, to voice (Gabai, column 
53, lines 26-36, . . . Their response includes, but is not limited to sound (including 
voice)... Fig. 66, 8085). 

Gabai fails to specifically teach classifying the inputted sentence as one of a 
plurality of types of sentences, the plurality of types of sentences including a question 
and an explanation; selecting a category of additional information related to the type of 
sentence, the category being an expression which is suitable to the type of sentence; 
selecting additional information in the selected category; and adding the additional 
information so that the additional information is added before the beginning of the 
inputted text or after the end of the inputted text. 

However, Papineni teaches classifying the inputted sentence as one of a plurality 
of types of sentences, the plurality of types of sentences including a question and an 
explanation (Papineni, column 15, lines 10-31 , the example shows input questions and 
explanations, see ... how about the vanguard index... (question), and ...i want to buy 
one hundred shares... (explanation). Additionally, column 15, lines 54-67 notes that the 
user asks "I would like to transfer five thousand dollars" (explanation), which includes 
the sentence "transfer five thousand dollars" which is further included in the response 
(as a confirmation question) below on lines 59-62.); 
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selecting a category of additional information related to the type of sentence, the 
category being an expression which is suitable to the type of sentence (Papineni, 
column 15, lines 19-31, shows selection of additional information related to the type of 
sentence through a confirmation, ...confirming purchase of one hundred Shares of 
vanguard index trust total stock market institutional shares, please say yes or no (fund 
name from context)... Column 15, lines 54-67 also show this confirmation of a transfer.); 
and 

selecting additional information in the selected category (Papineni, column 15, 
lines 54-67, selects additional information by providing specific confirmations of the 
index or fund, ...confirming transfer of five thousand dollars from fidelity magellan to 
vanguard index trust fund five hundred, please say yes or no. . . The selected additional 
information is the type of action and fund for the category of additional information 
(confirmation). Additionally, note the confirmation is in two forms, first being the 
"confirming transfer" at the beginning of the response, second being "please say yes or 
no" at the end of the response.); and 

adding the additional information to the inputted text with a change processing 
device so that the additional information is added before the beginning of the inputted 
text or after the end of the inputted text (Papineni, column 15, lines 54-67, ...confirming 
transfer of five thousand dollars from fidelity magellan to vanguard index trust fund five 
hundred, please say yes or no... The confirmation (additional information added to the 
input text of "transfer five thousand dollars") is in two forms, first being the "confirming 
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transfer" at the beginning of the response, second being "please say yes or no" at the 
end of the response.). 

Gabai and Papineni are analogous art because both deal with dialog interaction 
between a human and machine. Therefore, It would have been obvious to someone of 
ordinary skill in the art at the time of the invention to apply the 
confirmations/clarifications of Papineni with the base process of dialog interaction in 
Gabai because the result would have been predictable in providing a machine/user 
dialog interaction where the machine and user have a mutual understanding of the 
current dialog state. Therefore, it would have been obvious to someone of ordinary skill 
in the art at the time of the invention was made to combine Papineni with Gabai. 

As per claim 74, Gabai teaches a terminal (Gabai, column 16, lines 7-30) 
comprising: 

an information processing device for receiving inputted text, having an 
information changing unit for analyzing the sentence to determine information to be 
added comprising the steps of (Gabai, column 43, lines 20-34, A server (information 
processing device for receiving inputted text) may receive the scanned information to 
translate (information changing unit supplied by server) the message with greater 
computing power. Further, Gabai, column 43, lines 3-19, ...It is preferred, in such cases, 
that a toy not merely translate but combine its translations with other types of content 
that is appropriate to the given situation... Therefore, the translating unit may also add 
translation content information. Furthermore, column 43, lines 35-50 gives examples. 
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one of which is that the user inputs a menu and then toy subsequently explains the 
cultural significance of the dishes.): 

adding the additional information to text (Gabai, column 43, lines 3-19, ...It is 
preferred, in suet) cases, that a toy not merely translate but combine its translations with 
other types of content that is appropriate to the given situation. . .Furthermore, column 
43, lines 35-50 gives examples, one of which is that the user inputs a menu and then 
toy subsequently explains the cultural significance of the dishes. Figs .14 and 70 show 
that the output can be processed an generated from the internal toy process or an 
external computer/server depending on the complexity of the input/operation.); and 

an information reproducing device for converting and output from the information 
changing unit to voice (Gabai, column 53, lines 26-36, ...Their response includes, but is 
not limited to sound (including voice)...). 

Gabai fails to teach classifying the sentence as one of a plurality of types of 
sentences, the plurality of types of sentences including a question and an explanation; 
selecting a category of additional information related to the type of sentence, the 
category being an expression which is suitable to the type of sentence; and selecting 
additional information in the selected category and adding the information before the 
beginning of the inputted text or after the end of the inputted text; and an information 
reproducing device for converting an output from the information changing unit to voice. 

However, Papineni teaches classifying the sentence as one of a plurality of types 
of sentences, the plurality of types of sentences including a question and an explanation 
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(Papineni, column 1 5, lines 1 0-31 , the example shows input questions and 
explanations, see ... how about the vanguard index... (question), and ...i want to buy 
one hundred shares... (explanation). Additionally, column 15, lines 54-67 notes that the 
user asks "I would like to transfer five thousand dollars" (explanation), which includes 
the sentence "transfer five thousand dollars" which is further included in the response 
(as a confirmation question) below on lines 59-62.); 

selecting a category of additional information related to the type of sentence, the 
category being an expression which is suitable to the type of sentence (Papineni, 
column 15, lines 19-31, shows selection of additional information related to the type of 
sentence through a confirmation, ...confirming purchase of one hundred Shares of 
vanguard index trust total stock market institutional shares, please say yes or no (fund 
name from context)... Column 15, lines 54-67 also show this confirmation of a transfer.); 

selecting additional information in the selected category and adding the 
additional information before the beginning of the inputted text or after the end of the 
inputted text (Papineni, column 15, lines 54-67, selects additional information by 
providing specific confirmations of the index or fund, ...confirming transfer of five 
thousand dollars from fidelity magellan to vanguard index trust fund five hundred, 
please say yes or no... The selected additional information is the type of action and 
fund for the category of additional information (confirmation). Additionally, note the 
confirmation is in two forms, first being the "confirming transfer" at the beginning of the 
response, second being "please say yes or no" at the end of the response.); and 
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Gabai and Papineni are analogous art because both deal with dialog interaction 
between a human and machine. Therefore, It would have been obvious to someone of 
ordinary skill in the art at the time of the invention to apply the 
confirmations/clarifications of Papineni with the base process of dialog interaction in 
Gabai because the result would have been predictable in providing a machine/user 
dialog interaction where the machine and user have a mutual understanding of the 
current dialog state. Therefore, it would have been obvious to someone of ordinary skill 
in the art at the time of the invention was made to combine Papineni with Gabai. 

As per claim 75, Gabai teaches a server comprising (Gabai, abstract, teaches 
that the toy can use cellular technology which is well known in the art to be able to 
independently process input as well as process the input through a server.); 

a communication device for communicating with a terminal (Gabai, column 16, 
lines 1 8-31 teaches a connection link that establishes a data link over a network to the 
toy where it is at least partly wireless. Fig. 3 further describes how network connected 
servers may be linked to the toy. The servers require a communication device (e.g. 
wireless network card) for the wireless link.); 

an information processing device for translating text received through the 
communication device from first language to second language (Gabai, column 43, lines 
20-34, A server (information processing device for receiving inputted text) may receive 
the scanned information to translate (information changing unit supplied by server) the 
message with greater computing power. Gabai, column 43, lines 3-19, Figs. 58 A-B 
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teach that the toy interprets the scanned information in a language not native to the user 
for the user's understanding.); 

an information changing unit for analyzing the text translated to the second 
language, determining information to be added on the basis of the analysis result 
comprising the steps of (Gabai, column 43, lines 3-19, ...It is preferred, in such cases, 
that a toy not merely translate but combine its translations with other types of content 
that is appropriate to the given situation... Therefore, the translating unit may also add 
translation content information. Gabai teaches the ability to read local or ancient 
languages, column 43, lines 20-34 where the scanner is an information analysis unit. 
Information to be added is appropriate to a given situation.); 

adding the information to the text translation (Gabai, column 43, lines 3-19, ...It is 
preferred, in such cases, that a toy not merely translate but combine its translations with 
other types of content that is appropriate to the given situation. . . Furthermore, column 
43, lines 35-50 gives examples, one of which is that the user inputs a menu and then 
toy subsequently explains the cultural significance of the dishes. Figs .14 and 70 show 
that the output can be processed an generated from the internal toy process or an 
external computer/server depending on the complexity of the input/operation.); 

transmitting an output from information changing unit through the communication 
device (Gabai, column 43, lines 20-34 teaches that the server may do the translating 
because it has a higher computation ability. Further see Figs. 58 a-d where the toy acts 
as a translation device. The toy outputs the translation results and therefore the 
translations have been transmitted back to the toy through the communication device.) 
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Gabai fails to teach classifying tfie sentence as one of a plurality of types of 
sentences, the plurality of types of sentences including a question and an explanation; 
selecting a category of additional information related to the type of sentence, the 
category being an expression which is suitable to the type of sentence; and selecting 
additional information in the selected category and adding the information before the 
beginning of the text or after the end of the text. 

However, Papineni teaches classifying the sentence as one of a plurality of types 
of sentences, the plurality of types of sentences including a question and an explanation 
(Papineni, column 1 5, lines 1 0-31 , the example shows input questions and 
explanations, see ...how about the vanguard index... (question), and ...i want to buy 
one hundred shares... (explanation). Additionally, column 15, lines 54-67 notes that the 
user asks "I would like to transfer five thousand dollars" (explanation), which includes 
the sentence "transfer five thousand dollars" which is further included in the response 
(as a confirmation question) below on lines 59-62.); 

selecting a category of additional information related to the type of sentence, the 
category being an expression which is suitable to the type of sentence (Papineni, 
column 15, lines 19-31, shows selection of additional information related to the type of 
sentence through a confirmation, ...confirming purchase of one hundred Shares of 
vanguard index trust total stock market institutional shares, please say yes or no (fund 
name from context)... Column 15, lines 54-67 also show this confirmation of a transfer.); 
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selecting additional information in the selected category and and adding the 
information before the beginning of the text or after the end of the text (Papineni, 
column 15, lines 54-67, selects additional information by providing specific 
confirmations of the index or fund, ...confirming transfer of five thousand dollars from 
fidelity magellan to vanguard index trust fund five hundred, please say yes or no. . . The 
selected additional information is the type of action and fund for the category of 
additional information (confirmation). Additionally, note the confirmation is in two forms, 
first being the "confirming transfer" at the beginning of the response, second being 
"please say yes or no" at the end of the response.). 

Gabai and Papineni are analogous art because both deal with dialog interaction 
between a human and machine. Gabai provides the base process of translated dialog 
interactions between the toy and the user and Papineni provides 
confirmations/clarifications to a dialog process. Therefore, it would have been obvious 
to someone of ordinary skill in the art at the time the Invention was made to modify the 
translation processing with additional information (as taught by Gabai) with the dialog 
confirmation additions (as taught by Papineni) because the result would have been 
predictable in providing a translation machine/user dialog interaction where the machine 
and user have a mutual understanding of the translation. Therefore, it would have been 
obvious to someone of ordinary skill in the art at the time of the invention was made to 
combine Papineni with Gabai. 
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15. Claims 65 and 71 are rejected under 35 U.S.C. 103(a) as being unpatentable by 
Gabai et al. (US Patent #6773344 hereinafter Gabai) in view of Papineni et al. (US 
Patent #6246981 ) and further in view of Uwakubo. (US Patent #651 301 1 ). 

As per claim 65, claim 62 is incorporated and the combination of Gabai and 
Papineni fail to specifically teach further comprising analyzing reaction time of a target 
for which the voice is output and determining the information on the basis of the 
analysis result with the information analysis unit. 

However, Uwakubo teaches further comprising analyzing reaction time of a 
target for which the voice is output (Uwakubo, columns 7-8, lines 63-67 and 1-8, ...a 
time period is clocked in some times, from a time when a reaction is presented to the 
output unit 360 (to the user) to another time when the user starts action in response to 
the presented reaction. ..). 

and determining the information on the basis of the analysis result with an 
information analysis unit (Uwakubo, column 8, lines 21-31, ...generate reactions or 
suspends the generating of the reactions, based on instructions from the conversation 
manage unit 330... A reaction is generated based upon the reaction time of the user.). 

Uwakubo, Gabai, and Papineni are analogous art because all pertain to dialog 
interactions. Therefore, it would have been obvious to modify information analysis unit 
(as taught by the combination of Gabai and Papineni) with the conversation manage 
unit (with the clock unit in combination with the recognition unit) to analyze user reaction 
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time (as taught by Uwakubo) to present output in a desired natural manner in response 
to the user input to make natural conversation (Uwakubo, column 4, lines 10-19). 

As per claim 71 , claim 67 is incorporated and the combination of Gabai and 

Papineni fails to teach wherein the information changing unit analyzes reaction time of a 
target for which the voice is output and determines the information on the basis of the 
reaction time. 

However, Uwakubo teaches wherein an information changing unit analyzes 
reaction time of a target for which the voice is output and determines the information on 
the basis of the reaction time (Uwakubo, columns 7-8, lines 63-67 and 1-8, ...a time 
period is clocked in some times, from a time when a reaction is presented to the output 
unit 360 (to the user) to another time when the user starts action in response to the 
presented reaction. . . ). 

Uwakubo, Gabai, and Papineni are analogous art because all pertain to dialog 
interactions. Therefore, it would have been obvious to modify information changing unit 
(as taught by the combination of Gabai and Papineni) with the conversation manage 
unit (with the clock unit in combination with the recognition unit) to analyze user reaction 
time (as taught by Uwakubo) to present output in a desired natural manner in response 
to the user input to make natural conversation (Uwakubo, column 4, lines 10-19). 

1 6. Claims 76-77 are rejected under 35 U.S.G. 1 03(a) as being unpatentable by 
Papineni et al. (US Patent #6246981) in view of McAllister et al. (US Patent #6385584) 
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As per claim 76, Papineni teaclies tine metliod comprising analyzing inputted text 
with an information analysis device to determine information to be added (column 3, 
lines 8-19, ...A system for conversant interaction includes a recognizer for receiving and 
processing input information and outputting a recognized representation of the input 
information. A dialog manager is coupled to the recognizer for receiving the recognized 
representation of the input information, the dialog manager having task-oriented forms 
for associating user input information therewith, the dialog manager being capable of 
selecting an applicable form from the task-oriented forms responsive to the input 
information...), comprising the steps of; 

adding the additional information to the inputted text so that the additional 
information is added before the beginning of the inputted text or after the end of the 
inputted text (Papineni, column 15, lines 54-67, ...confirming transfer of five thousand 
dollars from fidelity magellan to vanguard index trust fund five hundred, please say yes 
or no... The confirmation (additional information added to the input text of "transfer five 
thousand dollars") is in two forms, first being the "confirming transfer" at the beginning of 
the response, second being "please say yes or no" at the end of the response.); and 

outputting the inputted text which the information is added with an information 
reproducing device (Papineni, column 15, lines 54-67 teaches the dialog interaction 
including the confirmation that is added to the input text. Further see column 1 , lines 22- 
30, . . . The input and output could be either text-oriented or speech-oriented. Speech- 
oriented systems have a speech recognition subsystem (speech-to-text system) and a 
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speech synthesis subsystem (text-to-speech system)... Column 15, lines 5-67 further 
teach input sentences. The information reproducing device is interpreted as the text to 
speech synthesizer.). 

Papineni fails to teach generating a random number; and selecting additional 
information that corresponds to the random numbers. 

However, McAllister teaches generating a random number (McAllister, column 4, 
lines 32-44); 

and selecting additional information that corresponds to the random numbers 
(McAllister, column 4, lines 32-44, the random numbers are scaled to the number of 
voice message variations available for a particular prompt); 

McAllister and Papineni are analogous art because both pertain to language 
generation. Therefore, it would have been obvious to one of ordinary skill in the art at 
the time the invention was made to modify the confirmation responses (as taught by 
Papineni, yes or no confirmations, column 15) with the random wording variation (as 
taught by McAllister) so that "random wording variations are included in the prompts as 
would be characteristic of a human operator." (McAllister, Column 2, lines 28-39) 

As per claim 77, Papineni teaches an information processing system (Papineni, 
column 7, lines 41 -54 teaches a computer having a processor and memory) comprising: 

An information processing device for receiving inputted text, having an 
information changing unit for analyzing the inputted text to determine information to be 
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added (column 3, lines 8-19, ...A system for conversant interaction includes a 
recognizer for receiving and processing input information and outputting a recognized 
representation of the input information. A dialog manager is coupled to the recognizer 
for receiving the recognized representation of the input information, the dialog manager 
having task-oriented forms for associating user input information therewith, the dialog 
manager being capable of selecting an applicable form from the tasl<-oriented forms 
responsive to the input information...), comprising the steps of; 

adding the additional information to the inputted text so that the additional 
information is added before the beginning of the inputted text or after the end of the 
inputted text (Papineni, column 15, lines 54-67, ...confirming transfer of five thousand 
dollars from fidelity magellan to vanguard index trust fund five hundred, please say yes 
or no... The confirmation (additional information added to the input text of "transfer five 
thousand dollars") is in two forms, first being the "confirming transfer" at the beginning of 
the response, second being "please say yes or no" at the end of the response.); and 

an information reproducing device for converting an output from the information 
changing unit to voice (Papineni, column 15, lines 54-67 teaches the dialog interaction 
including the confirmation that is added to the input text. Further see column 1 , lines 22- 
30, ... The input and output could be either text-oriented or speech-oriented. Speech- 
oriented systems have a speech recognition subsystem (speech-to-text system) and a 
speech synthesis subsystem (text-to-speech system)... Column 15, lines 5-67 further 
teach input sentences. The information reproducing device is interpreted as the text to 
speech synthesizer.). 
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Papineni fails to teacli generating a random number; and selecting additional 
information that corresponds to the random numbers. 

However, McAllister teaches generating a random number (McAllister, column 4, 

lines 32-44); 

and selecting additional information that corresponds to the random numbers 
(McAllister, column 4, lines 32-44, the random numbers are scaled to the number of 
voice message variations available for a particular prompt); 

McAllister and Papineni are analogous art because both pertain to language 
generation. Therefore, it would have been obvious to one of ordinary skill in the art at 
the time the invention was made to modify the confirmation responses (as taught by 
Papineni, yes or no confirmations, column 15) with the random wording variation (as 
taught by McAllister) so that "random wording variations are included in the prompts as 
would be characteristic of a human operator." (McAllister, Column 2, lines 28-39) 

Conclusion 

17. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. Refer to PTO-892, Notice of References Cited for a listing of 
analogous art. 

1 8. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to GREG A. BORSETTI whose telephone number is 
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(571)270-3885, (FAX: 571 -270-4885). The examiner can normally be reached on 
Monday - Friday (8am - 5pm Eastern Time). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, RICHEMOND DORVIL can be reached on 571-272-7602. The fax phone 
number for the organization where this application or proceeding is assigned is 571 - 
273-8300. 

Information regarding the status of an application may be obtained from the 

Patent Application Information Retrieval (PAIR) system. Status information for 

published applications may be obtained from either Private PAIR or Public PAIR. 

Status information for unpublished applications is available through Private PAIR only. 

For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 

you have questions on access to the Private PAIR system, contact the Electronic 

Business Center (BBC) at 866-217-9197 (toll-free). If you would like assistance from a 

USPTO Customer Service Representative or access to the automated information 

system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Greg A. Borsetti/ 
Examiner, Art Unit 2626 
Greg.Borsetti@uspto.gov 



